Aiming at preserving the privacy of Personal Health Record (PHR) in Smart health (S-health), an attribute-based access control scheme with verifiable outsourced decryption and delegation was proposed. Firstly, the Ciphertext-Policy Attribute-Based Encryption (CP-ABE) was used to realize fine-grained access control of PHR. Secondly, the most complicated decryption was outsourced to the cloud server, and the authorized agency was used to verify the correctness of Partial Decryption Ciphertext (PDC) returned by the cloud server. Then, based on the delegation method, the outsourcing decryption and authentication could be delegated to third-party users without revealing privacy by restricted users. Finally, the adaptive security of the proposed scheme was proved under the standard model. The theoretical analysis results show that the decryption of user side only needs to perform one exponential operation, so that the proposed scheme has strong security and practicability.